A 136601 


Research  Note  83-34 


(D 


Implications  for  Development  of  the 
Collective  Training  Information 
System  (CTIS) 

Thomas  D.  Scott  and  Ana  G.  Ekstrom 


Submitted  by 
James  A.  Thomas,  Chief 

ARI  FIELD  UNIT  AT  PRESIDO  OF  MONTEREY,  CALIFORNIA 

and 

Harold  F.  O'Neil,  Jr.,  Director 
TRAINING  RESEARCH  LABORATORY 


1 


*7*7*7 


Unclassified 


"P.  '-•Vj’.t;  u*j  .■  ;  77  ■; .»  ■■■  ■— 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  (Whan  Data  Entered) 


REPORT  DOCUMENTATION  PAGE 


'  REPORT  NUMBER 

ARI  Research  Note  83-34 


2.  GOVT  ACCESSION  NO 


TITLE  (mnd  Subtitle) 

IMPLICATIONS  FOR  DEVELOP NENT  OF  THE  COLLECTIVE 
TRAINING  INFORMATION  SYSTEM  (CTIS) :  A  SUMMARY 


AUTHORfaJ 


Thomas  D.  Scott,  AnaG.  Ekstrom 


PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 


ARI  Field  Unit  -  PERI-10 
PO  Box  5787 

Presidio  of  Monterey,  CA  93940 


CONTROLLING  OFFICE  NAME  ANO  ADDRESS 

US  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences 

5001  Eisenhower  Ave.,  Alexandria  VA  22333 


<•*.  MONITORING  AGENCY  NAME  A  ADDRESS^!!  dllter an  I  Kan  Controlling  OlHce) 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


3.  RECIPIENT’S  CATALOG  NUMBER 


5.  TYPE  OF  REPORT  A  PERIOD  COVERED 


S.  PERFORMING  ORG.  REPORT  NUMBER 


8.  CONTRACT  OR  GRANT  NUMBERfaJ 


10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  A  WORK  UNIT  NUMBERS 


2Q263743A794 


12.  REPORT  DATE 

September  1983 


13.  NUMBER  OF  PAGES 

15 


IS.  SECURITY  CLASS,  (of  tble  rapor 1) 

Unclassified 


ISa.  DECLASSIFICATION/ DOWNGRADING 
SCHEDULE 


IS  DISTRIBUTION  STATEMENT  (of  (Ala  Report) 


Approved  for  public  release;  distribution  unlimited 


DISTRIBUTION  STATEMENT  (of  the  mbmlrmct  entered  In  Slock  20,  If  rffffarant  from  Report) 


I*.  SUPPLEMENTARY  NOTES 


*  KEY  WORDS  ( Continue  an  rereree  mi  da  If  nacaaaa ry  an d  Identity  by  block  number) 

Information  System  Performance  Data 

Collective  Training  Quality  Control 

Needs  Assessment  CTIS 

Evaluation/Feedback 


20.  ABSTRACT  fCmtimio  am  rereree  af*  M  rmeoaeorj  amd  Identity  by  Aloe*  number) 

v1  In  order  to  make  a  preliminary  determination  of  components  of  a  Collective 
Training  Information  System,  the  Army  Training  Board  requested  that  the  Army 
Research  Institute  conduct  an  Information  Needs  Assessment.  A  series  of  semi- 
structured  interviews  were  conducted  with  training  managers,  executives,  and 
ARTBP  developers. 

2  ' 

Interviews  revealed  a  definite  need  for  feedback  from  field  users  of  ARTEP 
documents  to  the  developers  pertaining  to  the  quality  and  utility  of  their 


oo 


JAM  7* 


1473 


EDITION  OF  •  NOV  SS  IS  OBSOLETE 


Unclassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  fRwi  Dio  Entered) 


SCCUMTY  CLASSIFICATION  OF  THIS  PA0'i(Whm  Data  Kntata4) 


20.  Abstract  (oont.) 

products.  Needed  evaluation  feedback  by  field  units  at  various  levels  revealed  a 
number  of  issues  that  a  collection  information  system  would  need  to  take  into 
account.  This  report  presents  a  summary  of  these  issues  and  outlines  information 
system  requirements  and  methods  that  could  address  the  identified  issues. 


j  Acce 
NTIS 


Accession  For 


NTIS  GRA&I  X* 

DTIC  TAB  Q 

Unannounced  □ 

Justification _ 

By - 

Distribution/ 
Availability  Codes 
Avail  and/or 
Dlst  j  Special 


\S 

V  y 


unclassified 


$tCU*tTY  CLASSIFICATION  OF  THIS  PAGEr^h**  Dete  Entered) 


rr*  * 


TABLE  CP  CONTENTS 


Page 

Introduction  1 

General  Information  Requirements  2 

a.  Training  Developers  2 

b.  Training  Managers  3 

c.  Unit  Executives  4 

d.  Additional  Considerations  4 

Quality  Control  5 

Information  Analysis  and  Use  5 

Information  Access  6 

Manpower  and  Personnel  7 

Development  Issues  8 

a.  System  Development  8 

b.  Configuration  9 

Conclusions  9 

References  10 


iii 


-V./-. 


INTRODUCTION 


The  purpose  of  this  report  is  to  sunmarize  and  extend  the  Collective 
Training  Information  System  (CTIS)  work  conducted  for  ATB  over  the  past  year. 
The  primary  focus  is  on  developing  preliminary  concepts  for  the  CTIS,  partly 
based  on  information  needs  and  operational  considerations  of  three  potential  user 
groups  surveyed  by  ARI.  It  should  be  emphasized  that  in  many  cases  this  report 
goes  well  beyond  the  data  collected  during  the  needs  assessment  survey.  This 
approach  was  taken  in  order  to  provide  meaningful  discussions  of  the  implications 
of  the  survey  results  in  a  systems  development  context.  Also,  some  systems 
development  issues  which  did  not  emerge  from  the  survey  are  covered  here  insofar 
as  they  seemed  useful  in  defining  CTIS  requirements.  Wherever  it  seemed 
advisable,  potential  solutions  to  development  problems  have  been  proposed,  and 
seme  topics  in  need  of  further  examination  have  been  identified.  In  contrast  to 
the  survey  reports,  then,  this  document  is  relatively  narrow  in  scope,  concen¬ 
trates  on  CTIS  issues,  and  does  not  cover  many  of  the  job  and  ARTEP-related 
topics  included  in  the  survey. 

This  seminary  report  is  intentionally  brief  and  covers  the  more  salient 
development  issues  at  a  conceptual  level.  Detailed  discussions  of  technical 
issues  have  not  been  included,  and  indeed  are  not  appropriate,  until  consensus  on 
system  concepts  has  been  reached.  This  report,  then,  should  be  viewed  as  a 
preliminary  step  toward  establishing  requirements  and  priorities  for  the  CTIS 
development  process. 

Before  proceeding  to  the  main  topic,  let  us  briefly  outline  the  needs 
assessment  effort.  The  assessment  consisted  of  interviews  with  members  of  each 
of  three  potential  CTIS  user  groups:  collective  training  developers,  unit 
training  managers,  and  unit  executives.  Training  developers  consisted  of  service 
school  managers  and  action  officers  involved  in  developing  ARTEP  materials. 
Training  managers  were  battalion  and  company  commanders.  Executives  consisted  of 
brigade  commanders,  G-3s,  and  one  Commanding  General.  Interviews  were  semi- 
structured  and  covered  a  variety  of  topics  including  opinions  of  the  Army 
Training  and  Evalation  Program,  perceptions  of  potential  users'  jobs,  reporting 
requirements,  and  needs  for  training-related  information.  Of  special  interest 
here  are  the  examples  of  training  data  computer  printouts  which  were  given  to 
each  respondent  for  evaluation.  These  consisted  of  several  ARTEP-derived 
examples  each  in  a  somewhat  different  format,  level  of  aggregation,  or  categori¬ 
zation  scheme.  The  principal  ones  showed  percentages  of  ARTEP  standards  achieved 
aggregated  into  task/ sifc  task,  functional  area  (e.g.,  planning),  or  weapon  system 
categories.  Examples  were  presented  data  by  platoon,  by  battalion  or  by  platoon 
over  time  (trends).  Respondents  were  also  asked  to  comment  on  the  usefulness  of 
normative  data  presentations.  For  example,  data  from  a  given  unit  might  be 
presented  with  division  or  even  Army-wide  averages  for  comparison  purposes. 
Detailed  comments  by  respondents  are  covered  in  the  three  previous  reports  and 
will  not  be  presented  here.  Finally,  the  caveats  regarding  the  small  sample  size 
and  the  representativeness  of  the  sample,  which  were  made  in  the  three  previous 
needs  assessment  reports,  are,  of  course,  equally  applicable  to  the  inferences 
made  in  this  report. 


GENERAL  INFORMATION  REQUIREMENTS 


There  appeared  to  be  substantial  support  for  the  CTIS  concept,  but  the 
support  was  tenpered  by  reservations.  These  reservations  centered  on  the 
respondents'  prior  experiences  with  innovations  which  have  generated  increased 
burdens  without  yielding  comparable  benefits.  Such  concerns  tended  to  be 
pervasive,  and  if  not  effectively  countered,  they  could  develop  into  a  signifi¬ 
cant  implementation  barrier.  In  order  to  maximize  the  potential  for  success, 
early  developments  should  concentrate  on  those  aspects  which  help  primary  users 
on  their  daily  jobs  aid  which  require  the  least  amount  of  change  or  support 
within  the  user  organizations.  Moreover,  a  close  working  relationship  between 
the  user  and  the  developers  is  mandatory,  especially  in  the  early  stages  of 
development.  A  very  good  guideline  for  developing  and  implementing  innovations 
along  these  lines  is  contained  in  "A  Guide  to  Implementation  of  Training 
Products,"  by  Gray  and  Roberts^ ray.  The  respondents'  concerns  notwithstanding, 
most  saw  the  CTIS  as  having  significant  potential  for  increasing  the  amounts  and 
types  of  information  available  to  them,  and  for  improving  the  efficiency  of 
operations. 

Not  surprisingly,  reported  information  needs  differed  considerably  between 
the  three  user  groups.  However,  the  differences  between  operational  unit  users 
(i.e.,  training  managers  and  unit  executives)  differed  primarily  in  level  of 
detail  and  data  presentation  formats.  Training  developers,  on  the  other  hand, 
reported  wanting  different  types  of  information  than  user  units. 

a.  Training  Developers 

Respondents  indicated  a  need  for  information  on  the  users  perceptions  of  the 
quality  of  the  training  products  which  are  sent  to  the  field.  Without  such 
information,  they  contended,  improvements  to  existing  training  products  or 
development  of  high  quality  new  products  is  exceedingly  difficult.  Respondents' 
comments  indicate  that  highly  detailed  information  is  required.  For  example  ,  if 
unit  personnel  indicate  that  some  training  objectives  are  unclear,  training 
developers  will  need  to  know  which  specific  tasks,  conditions  or  standards  are 
problems,  and  the  users'  assessments  of  the  sources  of  the  problem  in  each  case. 
In  order  to  handle  this  kind  of  textual  information  CTIS  developers  should 
incorporate  the  capability  for'  the  input  and  analysis  of  free-format  information 
and  should  consider  developing  a  scheme  for  using  codes  to  flag  and  categorize 
common  types  of  problems. 

There  is,  at  present,  no  routine  mechanism  for  obtaining  information  from 
field  user  groups,  and  transmitting  that  information  to  the  appropriate  devel¬ 
opers.  It  is  apparent  that  such  a  mechanism  is  a  key  aspect  of  the  projected 
CTIS,  and  the  information  problem  must  be  solved  if  the  system  is  to  yield 
sitostantial  long-term  benefits  to  the  training  development  community. 

Although  training  developers  indicated  no  need  for  performance  data  per  se, 
some  of  those  data  could  help  to  identify  problems  with  fielded  training 
objectives.  For  example,  the  frequency  of  "not  evaluated,"  or  similar  entries 
could  give  developers  information  about  ARTEP  standards  which  are  difficult  to 
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tpply  in  the  field.  Data  on  the  frequencies  of  tasks  and  subtasks  evaluated 
during  ARTEP  exercises  could  provide  important  information  on  actual  unit 
training  priorities  and  enable  developers  to  focus  their  efforts  better.  Also, 
these  kinds  of  information  should  be  easier  to  obtain  than  unit  performance  data, 
as  they  are  generally  not  considered  sensitive,  and  impose  no  additional  data 
collection  burden  on  the  unit. 


b.  Training  Managers. 


Respondents  reported  that  they  need  information  which  would  help  them 
determine  the  strengths  and  weaknesses  of  their  units,  and  of  the  leaders  of 
their  units.  Although  ARTEP  task- based  displays  of  training  data  were  regarded 
favorably,  respondents'  comments  suggest  that  the  CTIS  should  incorporate  soft¬ 
ware  for  displaying  separately  those  data  bearing  on  leader  performance  f ran 
those  data  dealing  with  performance  of  the  unit  as  a  whole.  Moreover,  a 
relatively  fine  level  of  detail  is  required  to  develop  and  manage  effective  unit 
training  programs.  Broad  summaries  are  generally  not  very  useful  at  the 
battalion  level  except  for  reporting  to  higher  echelons.  Comments  suggest  that  a 
CTIS  software  which  assisted  managers  in  identifying  caDmonalties  of  and  reasons 
for  training  deficiencies  would  be  favorably  received.  In  general,  training 
managers  did  not  find  displays  of  training  data  in  terms  of  weapon-systems  or 
trends  to  be  very  useful.  One  of  the  reasons  that  trends  were  not  viewed  more 
favorably  was  the  impact  of  turbulence  and  turnover  on  training  activities.  Were 
the  personnel  problems  to  become  less  severe,  some  trend  data  could  be  useful  in 
evaluating  the  success  of  unit  training  programs  and  possibly  the  soundness  of 
unit  training  management  practices.  Thus,  CTIS  developers  should  plan  to  develop 
software  to  support  trend  analyses  in  the  future. 


Although  respondents  did  not  indicate  a  need  for  training  data  organized 
into  function  categories,  several  noted  that  some  data  organization  fully 
consistent  with  National  Training  Center  take-hcme  package  or  after  action  report 
organization  was  highly  desirable.  The  NTC  data  information  is  organized  into 
"seven  operating  systems"  which  are  similar  in  many  respects  to  the  functional 
area  categories,  but  there  are  currently  several  inconsistencies  between  the  two 
categorization  schemes.  These  differences  will  need  to  be  addressed  by  CTIS 
developers  and  the  NTC-TRADOC  personnel  in  order  to  develop  a  coumon  organiza¬ 
tional  scheme.  An  important  area  for  work  in  the  near-term  is  development  of  a 
systematic  method  for  translating  NTC  data  into  concrete  training  guidance  at  the 
lower  echelons  (i.e.,  company  and  platoon  levels).  Currently  there  are  no 
explicit  associations  between  ARTEP  tasks,  conditions  and  standards  and  the  NTC 
"operating  systems."  A  "crosswalk"  between  these  two  types  of  training  frame¬ 
works  could  enhance  the  usefulness  and  acceptance  of  the  CTIS  at  the  uiit  level. 

Respondents  consistently  complained  that  extensive  management  related 
paperwork  adversely  affects  the  time  they  are  able  to  devote  to  training 
activities.  This  finding  suggests  that  the  CTIS  developers  should  examine  the 
requirements  for  and  feasibility  of  establishing  an  interface  between  the  CTIS 
and  an  automated  aid  to  training  management. 


c.  Unit  Executives. 

Executive  respondents  reported  desiring  the  sane  general  types  of  informa¬ 
tion  as  the  training  managers.  However,  because  of  the  broader  nature  of  their 
responsibilities,  less  detailed  Information  is  needed.  For  example,  the  execu¬ 
tives  found  performance  information  organized  into  functional  and  subfunctional 
area  categories  potentially  useful  but  did  not  consistently  indicate  needs  for 
task/sub- task,  or  weapon-system  aggregated  data.  Like  the  training  managers, 
these  respondents  also  noted  that  some  data  organization  consistent  with  National 
Training  Center  will  be  needed.  Although  executive  respondents  generally 
preferred  broad  sinmaries  of  training  data,  some  indicated  that  they  would  prefer 
to  have  the  capability  for  reviewing  detailed  information.  This  finding  suggests 
that  the  unit  executive  component  of  the  CTIS  be  structured  hierarchically  with 
the  broader  summaries  as  the  display  "default  option"  and  more  detailed  data 
available  qpon  specific  query.  That  is,  when  an  executive  user  requests 
information,  the  highest  level  aggregation  would  be  displayed  first  and  progres¬ 
sively  finer  levels  of  detail  would  be  available  ipon  subsequent  requests. 

Respondents  did  not  lndioate  a  great  need  for  trend  data,  basically  for  the 
same  reasons  given  by  training  managers,  i.e.,  the  effects  of  personnel  turbu¬ 
lence  and  turnover.  Some  did,  however,  suggest  that  normative  data  could  be 
helpful  in  da  ter mining  how  realistic  their  expectations  regarding  the  effects  of 
training  are. 

d.  Additional  Considerations 

At  the  primary  user  level,  trainers  and  training  managers  are  usually  avare 
of  their  units'  idiosyncrasies.  But,  outside  the  unit,  performance  data  often 
becomes  difficult  to  interpret.  CTIS  developers  should  consider  incorporating 
information  into  the  data  base  which  would  help  external  users  to  accurately 
interpret  and  employ  unit  training  information.  Data  on  unit  characteristics 
(e.g. ,  turbulence  and  turnover  rates,  percentage  fill,  and  training  history), 
evaluator  experience,  exercise  conditions,  etc.  could  aid  CTIS  training  develop¬ 
ment  and  unit  executive  user  groups.  In  addition,  information  on  individual 
skill  requirements  for  ARTEP  training  objectives,  doctrinal  literature  bearing 
on  tactical  tasks,  alternative  training  methods  for  remedial  training,  etc .  could 
provide  valuable  assistance  to  training  development  users.  Compilation  of  such 
lists  is  not  particularly  complicated,  but  constructing  a  detailed  cross- 
reference  indexing  system  is  a  painstaking,  time  oo naming  process.  Fortunately, 
some  of  the  groundwork  has  been  laid  in  the  course  of  developing  the  Company 
Training  Plan  with  its  fairly  extensive  cross-references.  CTIS  developers  should 
be  able  to  apply  muoh  of  this  work  direotly  in  developing  the  CTIS  data  base  file 
structure.  Provisions  will  need  to  be  made  for  parallel  efforts  by  each  of  the 
Service  Schools.  Depending  on  the  timing  of  a  prototype  CTIS  implementation 
effort  in  these  institutions,  software  could  be  developed  to  support  construction 
of  or 088- reference  indexes. 


QUALITY  CONTROL 


Although  the  training  development  respondents  did  not  raise  quality  control 
issues  with  regard  to  their  own  operations,  the  CTIS  could  help  to  control  the 
quality  of  ARTEP  products.  The  CTIS  software  to  support  both  the  Collective 
Front-end  Analysis  (CFEA)  management  and  the  CFEA  itself  could  incorporate  the 
capability  to  create  audit  trails  for  each  collective  task,  thereby  helping  to 
insure  that  critical  development  steps  are  not  omitted.  Also,  criteria  which 
training  objectives  should  meet  and  common  deficiency  cues  could  be  incorporated 
in  the  CTIS  software  to  help  developers  improve  the  consistency  and  the  quality 
of  their  products. 

Many  respondents  expressed  concern  over  quality  control  issues,  and  were 
especially  skeptical  about  the  quality  of  performance  data  resulting  from 
tactical  field  exercises.  Among  the  reasons  cited  were  lack  of  experienced 
evaluators,  limited  amounts  of  time  for  evaluator  training  and  variations  in 
exercise  conditions  (weather,  force  ratios,  terrain,  etc.).  Although  such  issues 
are  mostly  outside  the  scope  of  the  CTIS  development  per  se,  if  not  ameliorated 
they  are  likely  to  undermine  the  credibility  of  the  system  and  adversely  affect 
the  extent  of  utilization.  In  cases  in  which  data  is  somewhat  suspect,  one 
partial  solution  is  to  document  probable  sources  of  anomalies,  thus  providing 
interpretive  aids  to  the  users.  For  example,  if  lack  of  evaluator  experience  is 
a  probable  contributor  to  poor  data  quality,  information  on  evaluator  ranks,  MOS, 
time- in-grade,  number  of  exercises  previously  evaluated  and  so  on,  can  help  the 
user  to  interpret  the  data  accurately.  As  noted  earlier,  other  interpretative 
aids  might  include  data  on  the  units  recent  training  history,  turbulence  and 
turnover,  percentage  fill,  exercise  conditions  and  training  devices  employed. 
Because  the  performance  data  is  not  expected  to  be  of  exceptionally  high  quality, 
CTIS  developers  should  anticipate  incorporation  of  as  much  capability  for  cross¬ 
checking  data  as  possible. 

Other  ARI-PGM  efforts  should  also  have  a  positive  influence  on  some  primary 
quality  control  problems.  For  example,  improved  guidance  on  evaluator  training 
and  evalution  methods  is  being  prepared  to  help  units  conduct  better  ARTEP 
evaluations,  and  work  in  support  of  the  USAIS  is  expected  to  result  in  more 
objective,  easily  applied  evaluation  standards. 


INFORMATION  ANALYSIS  AND  USE 

The  data  analysis  requirements  identified  by  respondents  are  mostly 
straightforward  sums,  percentages  and  averages.  Complicated  an  ail  y  tic  support 
software  is  not  required.  However,  as  mentioned  above,  training  developers  will 
probably  require  extensive  cross-referencing  capabilities,  as  well  as  capability 
for  manipulating  other  non- numerical  information.  There  are  currently  available 
a  variety  of  relational  data  base  management  systems  (DBMS)  that  could  be  used  to 
satisfy  such  requirements.  Which  of  these  is  selected  will  depend  mainly  on  the 
hardware  and  operating  system  on  which  the  CTIS  is  to  be  implemented,  the 
flexibility  required,  and  cost  considerations.  In  addition  to  the  DBMS,  training 
developers  may  need  other  kinds  of  software  to  support  their  efforts,  little  of 
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which  will  probably  be  commercially  available.  For  example,  managers  of  training 
development  activities  may  need  management  support  software  to  help  them  keep 
track  of  product  developments,  exercise  some  quality  control  functions,  produce 
periodic  reports,  estimate  work  requirements,  and  so  on.  Training  developers  may 
need  softtare  to  support  improved  CFEAs,  up-date  task  files,  identify  development 
probLems  and  to  facilitate  production.  Extensive  coordination  and  interviews 
with  the  Service  School  collective  training  developers  will  be  needed  to  deter¬ 
mine  firm  requirements. 

In  the  longer  term  the  CTIS  may  be  used  to  sip  port  senior  Army  executives, 
can  bat  and  doctrine  developers,  the  combat  modeling  camnunity  and  others.  Some 
of  these  users  are  likely  to  require  very  succinct  summaries  of  complex  data 
sets.  Meeting  such  needs  is  likely  to  require  development  of  extensive  and 
rather  complicated  data  analysis  softtare.  CTIS  developers  will  need  to  investi¬ 
gate  needs  for  such  analyses  as  the  users  become  identified  over  the  next  few 
years.  Currently,  however,  development  of  sophisticated  mathematical  data 
analysis  software  is  probably  not  a  pressing  requirement. 


INFORMATION  ACCESS 

Because  of  the  Amy  emphasis  on  combined  arms  tactical  training,  developers 
from  all  service  schools  will  probably  require  some  information  from  sister 
institutions.  Thus,  the  CTIS  will  need  to  be  designed  to  facilitate  rapid 
retrieval  of  data  base  contents  in  the  developer's  specialty  area  and  in  other 
areas  which  are  closely  related.  This  raises  the  question  of  which  users  will 
have  access  to  various  sections  of  the  CTIS  core  data  base,  and  whether  portions 
of  the  data  base  will  require  restricted  access.  It  seems  likely,  for  example , 
that  those  developers  concerned  with  intelligence,  emerging  systems,  etc.,  may, 
for  security  reasons,  require  restricted  access  to  parts  of  the  data  base.  In 
addition,  the  CTIS  core  data  base  should  be  protected  from  unauthorized  changes, 
and  some  software  will  be  needed  to  insure  that  changes  have  the  proper 
endorsements  before  the  system  will  allow  either  deletions  of  existing  informa¬ 
tion,  or  insertion  of  new  information.  One  partial  solution  to  the  data  base 
integrity  issue  would  be  to  acquire  a  data  base  management  system  capable  of 
creating  "virtual  file  working  structures,"  thus  obviating  requirements  for 
temporary  changes  to  core  files  during  day-to-day  operations.  However,  an 
authorization  code  system  will  need  to  be  developed  in  any  case,  as  changes  to 
core  files  are  inevitable. 

A  major  CTIS  development  consideration  involves  the  clearly  sensitive 
nature  of  training  data.  Potential  users  at  all  levels  expressed  deep  concern 
that  training  data  might  be  mis  interprets  d  or  misused.  One  solution,  of  course, 
is  to  construct  the  system  to  safeguard  the  confidentiality  of  all  training  data 
by  removing  any  unit  identifiers  fran  data  transmitted  to  higher  echelons,  and  by 
aggregating  data  into  categories  (e.g.,  functional  areas)  not  traceable  to 
specific  units.  One  proposal  consistent  with  current  procedures  in  many  units 
is  to  remove  unit  identifiers  from  data  which  is  transmitted  more  than  two 
echelons  above  the  mlt  of  origin.  Thus,  brigade  headquarters  would  be  able  to 
identify  specific  companies  and  the  division  could  identify  units  only  down  to 


the  battalion  level.  The  central  idea  is  to  leave  unit  identifiers  intact  where 
they  materially  assist  managers  in  setting  training  priorities  and  in  allocating 
resources. 

At  the  same  time  CTIS  developers  should  be  aware  of  significant  moves  in  the 
Army  toward  greater  training  accountability.  On  the  one  hand,  safeguarding  data 
confidentiality  drastically  reduces  the  potential  for  misuse  while,  on  the  other 
hand,  it  virtually  eliminates  the  use  of  such  data  for  accountability  purposes. 
For  a  number  of  technical  reasons  as  well  as  the  possibility  of  a  shift  in 
training  accountability  policy  in  future  years,  CTIS  developers  should  insure 
that  the  relevant  software  is  designed  in  modular  formats  to  facilitate  changes. 


MANPOWER  AND  PERSONNEL 

Respondents  consistently  reported  personnel  and  manpower  problems.  In 
training  development  activities,  new  personnel  must  frequently  be  trained, 
leading  to  quality  control  and  production  problems.  Also,  operational  units  are 
rarely  full  strength,  and  turnover  and  turbulence  tend  to  be  severe.  Therefore, 
the  CTIS  should  be  designed  with  a  view  toward  minimizing  the  system  personnel 
requirements,  both  in  terms  of  numbers  of  people  required  for  operations,  and  in 
terms  of  the  data  processing  experience  and  expertise  needed  for  effective  use. 
The  CTIS  software  design  should  incorporate  as  many  "user  friendly"  functional 
routines  as  possible  including  menu  operations,  "help"  commands,  and  so  on. 
Also,  user  instructional  courseware  could  be  developed  both  to  help  users 
Interact  mere  effectively  with  the  system,  and  to  provide  orientation  to  and 
basic  instruction  on  the  requirements  of  their  jobs. 

In  addition,  the  feasibility  of  automating  some  functions  which  are  current¬ 
ly  executed  manually  should  be  examined.  For  example,  much  of  the  training 
development  production  work  could  be  added  by  the  CTIS.  Software  to  support 
formatting,  printing  camera-ready  copy,  final  editing,  preparation  of  production 
status  reports,  etc.,  could  be  developed  for  the  CTIS.  Since  many  commercial 
printers  have  changed  from  manual  to  computer-supported  production  systems,  it  is 
likely  that  some  off-the-shelf  software  to  support  document  production  may  be 
available.  Likewise,  the  CTIS  could  be  used  to  reduce  units'  manpower  require¬ 
ments  by  providing  software  and  hardware  to  organize  and  print  ARTEP  exercise 
data  collection  forms,  to  reduce  and  analyze  training  data,  and  to  prepare 
reports  based  on  those  data. 

As  noted  earlier,  the  respondents  complained  consistently  that  they  were 
often  overburdened  with  paperwork.  The  CTIS  developers  could  examine  the 
feasibility  of  partially  automating  the  field  data  collection  process  through  the 
use  of  hand-held  data  collection  devices.  One  approach  is  to  load  such  devices 
with  training  objectives  from  a  CTIS  terminal  interface,  use  the  devices  for 
field  data  collection,  and  then  off-load  the  information  from  the  devices  back 
onto  the  CTIS  terminal  for  analysis  and  report  preparation.  Such  an  extended 
CTIS  terminal  could  significantly  reduce  units'  manpower  requirements  for  han¬ 
dling  training  data.  This  is  a  long-term  effort  and  exploratory  work  would  need 
to  be  undertaken  in  the  near-term  in  order  to  realize  the  benefits  2-3  years  from 


High  rates  of  turbulence  indicate  that  feedback  must  be  immediate  to  be  of 
any  use  at  squad,  platoon,  and  company  levels.  At  these  echelons  feedback  should 
be  very  specific,  sib- task  oriented,  and  presented  in  a  readily  understandable 
format.  In  order  to  facilitate  the  incorporation  of  ARTEP  feedback  into 
immediate  planning  for  corrective  training,  CTIS  should  be  integrated  with 
whatever  system  is  developed  to  assist  battalion  training  management. 


DEVELOPMENT  ISSUES 

There  are  a  variety  of  issues  which  should  be  addressed  early  in  the  CTIS 
development.  These  are,  for  the  most  part,  beyond  the  scope  of  the  user  group 
surveys,  but  are  nevertheless  central  to  developing  a  workable  CTIS  design 
cone  ept . 

a.  System  Development.  An  integrated  plan  for  CTIS  development  is  central 
to  insuring  that  the  operational  system  will  meet  user  needs,  and  that  it  is 
developed  as  cost-effectively  as  possible.  Because  of  the  broad  scope  of  the 
effort  the  current  information  suggests  that  the  system  should  be  developed  and 
implemented  in  phases.  Initial  efforts  would  focus  on  developing  a  system  for 
training  developers,  while  later  phases  would  focus  on  a  CTIS  for  operational 
units,  senior  executives,  combat  developers,  doctrine  developers  and  probably 
other  users  not  yet  identified. 

A  considerable  amount  of  softie  re  will  be  needed  to  implenent  the  CTIS. 
Host  often,  software  developed  for  one  computer  system  must  be  at  least  modified, 
and  occasionally  rewritten  entirely,  in  order  to  run  on  other  systems.  It  is 
likely  this  will  be  the  case  with  the  initial  system  developed  for  the  Burroughs 
and  a  later  one  developed  for  the  AMDAHL.  Because  such  softie  re  developments  are 
often  time  consuming  and  expensive,  CTIS  developers  need  to  consider  carefully 
the  effects  of  producing  software  not  readily  transferable  to  the  "end-product" 
system. 

The  initial  tryout  of  the  CTIS  should,  of  course,  be  made  with  a  limited 
prototype.  Careful  consideration  will  need  to  be  made  of  what  portions  of  the 
whole  software  package  for  each  phase  most  require  extensive  user  testing.  In 
addition,  it  may  be  valuable  to  develop  some  limited  programs  to  support  the  user 
tests.  For  example  some  simple  programs  which  count  the  number  of  times  each 
display  is  retrieved,  the  duration  of  that  display  is  held  on  the  screen,  the 
sequences  in  which  displays  are  retrieved,  etc.  could  greatly  assist  in  determin¬ 
ing  whether  the  system  is  being  used  as  intended,  and  what  kinds  of  changes  would 
be  advisable  in  the  final  version. 

At  some  point,  the  CTIS  will  probably  be  integrated  with  the  other  ATIS 
subsystems.  The  nature  of  the  interfaces  and  the  requirements  for  transmitting 
data  across  the  "subsystem  boundaries"  have  not  yet  been  determined.  While  the 
Independence  of  the  ATIS  subsystems  permits  relatively  rapid  development  of  each 
one,  this  approach  raises  fundamental  questions  regarding  compatibility  among 
the  subsystems.  If  developments  for  the  ATIS  subsystems  proceed  without  unified 
technical  management,  the  ultimate  integration  could  prove  extremely  awkward  and 
ooatly. 
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b.  Configuration.  It  is  expected  that  the  system  configuration  will  vary 
somewhat  with  the  various  users.  The  nunber  of  terminals  required,  graphics 
needs,  maximum  response  times,  and  so  on  should  be  determined  as  early  as 
possible.  In  addition,  the  location  or  locations  of  system  data  base(s)  can  be 
expected  to  have  a  major  impact  on  requirements.  For  example,  a  centralized  data 
base  would  allow  the  colocation  of  many  peripherals,  (e.g.,  the  disc-drive 
systems  which  tend  to  be  expensive).  Distributed  data  bases,  on  the  other  hand, 
require  distributed  data  processing  equipment  and  would  probably  entail  a 
substantial  amount  of  hardware  duplication.  The  trade-offs  between  system  costs, 
response  time  requirements,  access  restrictions,  etc.,  need  to  be  thoroughly 
examined  in  order  to  produce  a  viable  system  concept. 


CONCLUSIONS 

There  appears  to  be  sibstantial  support  for  the  CTIS  concept.  The  respon¬ 
dents,  however,  did  express  some  reservations  about  the  requirements  for  support¬ 
ing  such  a  system.  CTIS  developers  will  need  to  demonstrate  to  the  users  not  only 
the  benefits  to  be  derived  from  implementing  the  system,  but  also  that  the  oosts 
(manpower,  funds,  etc.)  born  by  the  users  are  not  excessive. 

Respondents  from  operational  units  found  the  sample  printouts  to  be  poten¬ 
tially  quite  useful  though  they  differed  considerably  in  their  ideas,  what  types 
of  data  would  be  most  useful,  how  best  to  organize  them,  and  what  levels  of 
aggregation  would  be  mast  desirable.  Training  development  respondents,  on  the 
other  hand,  did  not  find  the  examples  particularly  useful,  mainly  because  they 
require  information  on  fielded  ARTEP  materials  rather  than  data  on  unit  perfor¬ 
mance.  Nevertheless,  one  might  suggest  that  some  performanc e- related  data  could 
be  analyzed  in  ways  which  might  help  in  the  identification  of  problems  with 
training  objectives.  This  topic  should  be  explored  further. 

Current  plans  call  for  the  CTIS  to  be  developed  in  three  phases,  each 
servicing  the  needs  of  one  primary  user  group.  Phase  I  will  concentrate  on  ARTEP 
developers;  Phase  II,  operational  units;  and  Phase  III,  senior  Army  leaders, 
combat  and  doctrine  devriopers,  DA  staff  Elements,  etc.  The  plan  for  phased  CTIS 
development  was  formulated  after  the  survey  was  completed,  and  has  not  been  a 
focus  of  the  present  report.  The  CTIS  Phase  I  will,  however,  be  the  principal 
topic  of  the  preliminary  design  concept  report,  currently  scheduled  to  be 
completed  later  this  year. 
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